Field of the Invention
The present invention relates to an image processing apparatus, an image processing method, and a computer-readable storage medium.
Description of the Related Art
Conventionally known is changing a resolution between an input image and an output image. For example, image conversion processing called reduction processing or enlargement processing is known to be performed to obtain a recording resolution different from an image capturing resolution or display a broadcast moving image signal on a display device whose resolution is different from the resolution of the signal. In addition, image conversion processing called geometric transformation processing is known to be performed to allow a projector to project an image from an oblique direction or onto a curved screen. In a digital camera, a digital video camera, a flatbed scanner, or the like, conventionally, an optically formed image of an object is converted into a digital image by a digital image sensor. These image conversion processes will generically be referred to as conversion processing hereinafter.
When performing image transformation processing out of the conversion processes, a disturbance (noise) called a moiré sometimes occurs depending on the contents of an image. The moiré, which is also called interference fringes, is a beat phenomenon of a spatial frequency that occurs between a sampling frequency and a repetitive pattern in an image. The moiré is very noticeable and needs to be prevented to maintain image quality. When performing digital imaging processing out of the conversion processes as well, a moiré may occur depending on the contents of an object, and the moiré needs to be prevented. If the RGB elements of the digital image sensor have an arrangement called a Bayer arrangement, not only a moiré but also a disturbance called a false color may occur, and the false color also needs to be prevented.
For example, in reduction processing including geometric transformation processing, a moiré is known to be prevented by decreasing a high-frequency region using a low-pass filter (to be abbreviated as an LPF hereinafter) applied to an input image and then performing reduction processing including geometric transformation processing. Since the image is blurred by this processing, high-frequency region enhancement processing is sometimes performed at the subsequent stage. In addition, for example, a moiré and false colors are known to be prevented by decreasing the high-frequency region of an object using an optical LPF arranged in front of an image sensor and then performing digital imaging.
To prevent a moiré or blur, Japanese Patent Laid-Open No. 3-40179 describes enhancing the high-frequency region of an input image in advance and removing a moiré generated after reduction processing. In this arrangement, the high-frequency region degrades little. Japanese Patent Laid-Open No. 2012-60517 describes changing the intensity of an LPF in accordance with an image reduction ratio. In this arrangement, high-frequency components of an image which exceed the Nyquist frequency upon reduction processing are removed by the LPF, thereby preventing a moiré. Japanese Patent Laid-Open No. 2-244876 describes divisionally transforming low-frequency components and high-frequency components of an image and performing edge enhancement only for the high-frequency components to prevent a blur caused by reduction. According to this arrangement, the blur of the high-frequency components caused by reduction can be prevented to some extent. Japanese Patent Laid-Open No. 2006-80845 describes combining a plurality of images obtained by performing digital imaging processing a plurality of times while changing the characteristic of an optical LPF such that portions with a moiré or false color are replaced. This can prevent a moiré or false color.
When transformation processing is performed for an image that has passed through a low-pass filter, and high-frequency enhancement processing is then performed, the lost information of the high-frequency region cannot be restored by the enhancement. Hence, the image quality degrades.
When enhancing the high-frequency region of an input image in advance and removing a moiré after reduction processing, as in the arrangement of Japanese Patent Laid-Open No. 3-40179, a low-pass filter such as a moving filter is applied to a moiré frequency to remove the moiré. In this case, image components of the same frequency as the moiré frequency decrease, and the image quality of low-frequency components degrades.
In the arrangement of Japanese Patent Laid-Open No. 2012-60517, high-frequency components of an image which exceed the Nyquist frequency upon reduction processing are removed. This method can effectively be used to remove a moiré generated by the high-frequency components higher than the Nyquist frequency but is not necessarily effective in a case in which an image is enlarged or in a case in which a moiré is generated by interference between a sampling frequency and a fringe pattern in an image at the time of image reduction. In addition, a blur remains in a portion where the image reduction ratio is high.
When divisionally transforming low-frequency components and high-frequency components of an image and performing edge enhancement only for the high-frequency components to prevent, as in the arrangement of Japanese Patent Laid-Open No. 2-244876, a moiré generated in the high-frequency components remains even after the edge enhancement. For this reason, a moiré occurs in the output image. Assume that two images of different intensities are obtained by changing the characteristic of an optical LPF and combined using the image of the weak optical LPF as a seed such that only portions with a moiré or false color are replaced with the image of the strong optical LPF, as in Japanese Patent Laid-Open No. 2006-80845. In this case, the image has neither a moiré nor a false color but has a blur in the replaced portions.